R1 → R1 + bR3 (i.e. replace 1^st row by addition of 1^st row and b times 3^rd row)

R2 → R2 - aR3 (i.e. replace 2^nd row by subtraction of 2^nd row and a times 3^rd row)

.

Taking both (1 + a² + b²) outside the determinant from 1^st and 2^nd row

Expanding the determinant along 1^st row

∴ LHS = (1 + a² + b²)² [1 + a² - b² - (-2b²)]

∴ LHS = (1 + a² + b² )² (1 + a² + b²)

∴ LHS = (1 + a² + b² )³ = RHS